In this project, we will generate top 10 movie recommendations for each user as well as generate top 10 user recommendations for each movie. Welcome to this project on creating Movies Recommendation Engine using Apache Spark Machine Learning using Databricks platform community edition server which allows you to execute your spark code, free of cost on their server just by registering through email id. In this project, we explore Apache Spark and Machine Learning on the Databricks platform. I am a firm believer that the best way to learn is by doing. That's why I haven't included any purely theoretical lectures in this tutorial: you will learn everything on the way and be able to put it into practice straight away.

Recommendation has been a long-standing problem in many areas ranging from e-commerce to social websites. Most current studies focus only on traditional approaches such as content-based or collaborative filtering while there are relatively fewer studies in hybrid recommender systems. Due to the latest advances of deep learning achieved in different fields including computer vision and natural language processing, deep learning has also gained much attention in Recommendation Systems. There are several studies that utilize ID embeddings of users and items to implement collaborative filtering with deep neural networks. However, such studies do not take advantage of other categorical or continuous features of inputs. In this paper, we propose a new deep neural network architecture which consists of not only ID embeddings but also auxiliary information such as features of job postings and candidates for job recommendation system which is a reciprocal recommendation system. Experimental results on the dataset from a job-site show that the proposed method improves recommendation results over deep learning models utilizing ID embeddings.

The overwhelming volume and complexity of information in online applications make recommendation essential for users to find information of interest. However, two major limitations that coexist in real world applications (1) incomplete user profiles, and (2) the dynamic nature of user preferences continue to degrade recommender quality in aspects such as timeliness, accuracy, diversity and novelty. To address both the above limitations in a single solution, we propose a novel cross-network time aware recommender solution. The solution first learns historical user models in the target network by aggregating user preferences from multiple source networks. Second, user level time aware latent factors are learnt to develop current user models from the historical models and conduct timely recommendations. We illustrate our solution by using auxiliary information from the Twitter source network to improve recommendations for the YouTube target network. Experiments conducted using multiple time aware and cross-network baselines under different time granularities show that the proposed solution achieves superior performance in terms of accuracy, novelty and diversity.

Social tagging systems have emerged as an effective way for users to annotate and share objects on the Web. However, with the growth of social tagging systems, users are easily overwhelmed by the large amount of data and it is very difficult for users to dig out information that he/she is interested in. Though the tagging system has provided interest-based social network features to enable the user to keep track of other users' tagging activities, there is still no automatic and effective way for the user to discover other users with common interests. In this paper, we propose a User Recommendation (UserRec) framework for user interest modeling and interest-based user recommendation, aiming to boost information sharing among users with similar interests. Our work brings three major contributions to the research community: (1) we propose a tag-graph based community detection method to model the users' personal interests, which are further represented by discrete topic distributions; (2) the similarity values between users' topic distributions are measured by Kullback-Leibler divergence (KL-divergence), and the similarity values are further used to perform interest-based user recommendation; and (3) by analyzing users' roles in a tagging system, we find users' roles in a tagging system are similar to Web pages in the Internet. Experiments on tagging dataset of Web pages (Yahoo!~Delicious) show that UserRec outperforms other state-of-the-art recommender system approaches.